Portable information terminal and image management program

ABSTRACT

A portable information terminal and an image management program are provided. A portable information terminal includes: a display part which has a screen capable of displaying at least an image; an in-focus value computing part which computes an in-focus value from an original image; an image cutting part which cuts a thumbnail image from the original image based on the in-focus value computed at the in-focus value computing part; a data storage part which stores data of the thumbnail image cut at the image cutting part; and a control part which reads the data of the thumbnail image stored in the data storage part as necessary and displays it on a screen of the display part.

CROSS REFERENCES TO RELATED APPLICATIONS

This is a Continuation Application of U.S. patent application Ser. No.11/523,624, filed Sep. 20, 2006, which in turn claims priority fromJapanese Application No.: 2005-278724, filed on Sep. 26, 2005, theentire contents of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a portable information terminal whichcan show an image list on a display device mounted thereon, and an imagemanagement program which creates a thumbnail image from an originalimage and displays it in a list.

2. Description of the Related Art

Before, in an image list representation application, a taken image or animage stored in a storage is scaled down, and is formed in a thumbnailimage, and thumbnail images are listed and displayed on the screen of adisplay device.

In addition, in the case in which thumbnail images are listed anddisplayed on the screen of a display device mounted on a cellulartelephone terminal, the size and resolution of thumbnail images are alsoscaled down in accordance with the screen size and resolution of thedisplay device, and thus the images are really difficult to see.

Here, in the Publication of unexamined patent applicationJP-A-2000-224465 (Patent Reference 1), a technique is disclosed in whicha digital still camera is provided with a sight-line detecting partwhich detects the convergent point of the sight of a user, the essentialarea in a taken image is specified based on information about theconvergent point of the sight of the user detected by the sight-linedetecting part, that is, it is made possible to specify on which area inthe taken image the photographer focuses, and such a process isperformed that the resolution is increased or the compression ratio isreduced with respect to the image of the essential area, whereby animage easy to see can be created and on which the photographer'sintention is reflected.

In addition, in the Publication of unexamined patent applicationJP-A-2004-328283 (Patent Reference 2), a technique is disclosed in whicha portion is calculated at which a focal point is obtained in an image,the compression ratio of the portion at which the focal point isobtained is more reduced than the compression ratio of the otherportions, and whereby the deterioration of image quality is decreased,which is caused by scaling down an original image.

Patent Reference 1: JP-A-2000-224465 (FIGS. 1, 2 and 5)

Patent Reference 2: JP-A-2004-328283 (FIGS. 1 and 2)

SUMMARY OF THE INVENTION

However, it is difficult to adapt the technique described in thePublication of unexamined patent application JP-A-2000-224465 to acellular telephone terminal, for example. More specifically, forexample, in the technique described in the publication, the constructionof the sight-line detecting part which detects the convergent point ofsight of the user is necessary, and it is difficult to mount suchconstruction on the cellular telephone terminal. In addition, eventhough the technique described in the publication is adapted to improvethe image quality of the essential area on which the photographerfocuses, images listed on a small screen of a display device mounted onthe cellular telephone terminal are not made easy to see so much. Inaddition to this, for example, since the convergent point of sight isunknown in the case of an image obtained from outside, the essentialarea itself cannot be specified.

Similarly, also in the case in which the technique described in thePublication of unexamined patent application JP-A-2004-328283 is adaptedto the cellular telephone terminal, for example, it is difficult to makeimages easy to see that are listed on the screen of the display deviceof the cellular telephone terminal. More specifically, the techniquedescribed in the publication can suppress the deterioration of imagequality in scaling down the original image, but it cannot make imageseasy to see so much that are listed on a small screen of the displaydevice mounted on the cellular telephone terminal.

It is desirable to provide a portable information terminal and an imagemanagement program which allow images to be made easy to see that arelisted on the screen of a display device, such as a display devicemounted on a cellular telephone terminal that the screen size is smalland the resolution is not so high.

A portable information terminal according to an embodiment of theinvention solves the above problems by providing a portable informationterminal including: a display part which has a screen capable ofdisplaying at least an image; an in-focus value computing part whichcomputes an in-focus value from an original image; an image cutting partwhich cuts a thumbnail image from the original image based on thein-focus value computed at the in-focus value computing part; a datastorage part which stores data of the thumbnail image cut at the imagecutting part; and a control part which reads the data of the thumbnailimage stored in the data storage part as necessary and displays it on ascreen of the display part.

In addition, an image management program according to an embodiment ofthe invention solves the problems by providing an image managementprogram which allows a portable information terminal to function as: anin-focus value computing part which computes an in-focus value from anoriginal image; an image cutting part which cuts a thumbnail image fromthe original image based on the in-focus value computed at the in-focusvalue computing part; a storage control part which stores data of thethumbnail image cut out at the image cutting part in a data storagepart; and a display control part which reads the data of the thumbnailimage stored in the data storage part as necessary and displays it on ascreen of a display part.

More specifically, the in-focus area in the original image is highlylikely to be a target for shooting when an image is taken, and is thecharacteristic portion in the original image. In an embodiment of theinvention, the in-focus area (the characteristic portion in the originalimage) is cut out of the original image as a thumbnail image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a diagram depicting an application stack of a cellulartelephone terminal according to an embodiment of the invention;

FIG. 2 shows a diagram depicting an exemplary taken original image;

FIG. 3 shows a diagram depicting a state that an original image isdivided into a plurality of equal areas and an exemplary thumbnail imagecut out of the original image;

FIG. 4 shows a diagram depicting an exemplary three-dimensional in-focusvalue distribution obtained from in-focus computation according to amodified Laplacian method;

FIG. 5 shows a diagram depicting an exemplary thumbnail image cut out ofthe original image;

FIG. 6 shows a flow chart depicting the flow of a thumbnail imagecreating process according to the embodiment; and

FIG. 7 shows a block diagram schematically depicting the internalconfiguration of the essential part of the cellular telephone terminalaccording to the embodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

Hereinafter, an embodiment of a portable information terminal and animage management program will be described with reference to thedrawings.

In addition, in the embodiment, a cellular telephone terminal is takenas an exemplary portable information terminal according to an embodimentof the invention. However, of course, the descriptions explained hereare only examples, and it is without saying that the invention is notlimited to these examples.

A cellular telephone terminal according to the embodiment is a terminalinstalled with a camera function, and the camera function provides afunction that detects the area in which a focal point is obtained(hereinafter, denoted as an in-focus area) for a taken image or an imageobtained from outside, cuts the in-focus area thereof out of an originalimage, and creates a thumbnail image from the cut in-focus area.

Application Stack

FIG. 1 shows an application stack of the cellular telephone terminalaccording to the embodiment.

In FIG. 1, the cellular telephone terminal according to the embodimenthas a camera application 10 and a viewer application 11 as anapplication layer.

The camera application 10 is the application that uses the camerafunction provided in the cellular telephone terminal to shoot an image.The camera application 10 is executed to take image data, and the takenimage data are stored as an original image 14 in data storage 13 forimage data through an image managing part 12 of a middleware. Inaddition, the data storage 13 is configured to allow storing the imagetaken by the camera function as well as storing an image obtained fromoutside, for example, as the original image 14.

The viewer application 11 is the application that executes a browsingprocess in which the image taken by the camera application 10, the imageobtained from outside, and the thumbnail image created by the imagemanaging part 12, described later, are displayed on the screen of adisplay device provided in the cellular telephone terminal according tothe embodiment.

The image managing part 12 exists in the middleware layer, which is thesoftware that runs the image management program according to anembodiment of the invention in which in storing the original image 14taken by the camera function and the original image 14 obtained fromoutside in the data storage 13, as described later, the in-focus area isdetected based on the data of the original image 14 as well as thein-focus area is cut out to create a thumbnail image 15 from the cutin-focus area. Subsequently, the image managing part 12 stores thethumbnail image 14 in the data storage 13.

In addition, for example, when the viewer application 11 requests thereading of the thumbnail image, the image managing part 12 reads thethumbnail image 14 stored in the data storage 13, and supplies it to theviewer application 11. Thus, on the screen of the display device mountedon the cellular telephone terminal according to the embodiment, thethumbnail image 14 is listed.

Thumbnail Image Creating Process

Hereinafter, a thumbnail image creating process performed in the imagemanaging part 12 will be described.

The camera function of the cellular telephone terminal according to theembodiment has a focus control mechanism. Suppose that the focus controlmechanism obtains the focal point, and then an original image 20 istaken as shown in FIG. 2, for example. In addition, suppose that amongsome of subject images 21 to 23 (hereinafter, they are referred to asobjects 21 to 23) shown in the original image 20, the target subjectthat a photographer intends to shoot, that is, the subject on which thefocal point is obtained by the focus control mechanism is the object 21.The other objects 22 and 23 are images out of focus as compared with theobject 21 which is the focal point obtained for shooting.

In this case, as shown in FIG. 3, the image managing part 12 accordingto the embodiment divides the original image 20 into a plurality ofequal areas 28, and performs predetermined in-focus computation for eachof the areas 28 to compute the area in which the focal point isobtained. For the scheme of in-focus computation, there are variousschemes. In the embodiment, a modified Laplacian method is used, whichis a typical scheme thereof. More specifically, the modified Laplacianmethod is the scheme in which the focused pixel is expressed by 1 (x, y)to perform a computation as expressed by an Equation (1), whereby anin-focus value is determined.

$\begin{matrix}{{{Equation}{\mspace{11mu}\;}1}\mspace{625mu}} & \; \\{{\Delta_{M}^{2}{I( {x,y} )}} = {{\frac{\delta^{2}I}{\delta\; x^{2}}} + {\frac{\delta^{2}I}{\delta\; y^{2}}}}} & (1)\end{matrix}$

Of course, in the embodiment of the invention, methods other than themodified Laplacian method may be used.

Here, suppose that as the result of the in-focus computation accordingto the modified Laplacian method for each of the areas 28 of theoriginal image 20 shown in FIG. 3, for example, a three-dimensionalin-focus value distribution is obtained as shown in FIG. 4. Morespecifically, from the original image 20 shown in FIG. 3, such anin-focus value distribution is obtained that the in-focus value is greatin the area including the object 21 that the focal point has obtainedfor shooting whereas the in-focus value is small in the other areas.

Subsequently, the image managing part 12 compares the in-focus value foreach of the areas 28 with a predetermined threshold based on thein-focus value distribution as shown in FIG. 4, and determines only thearea in which the in-focus value exceeds the threshold as the in-focusarea as well as it cuts the image in a predetermined size including thedetermined in-focus areas as the thumbnail image. Alternatively, itdetects the area having the maximum in-focus value as the in-focus areaamong the individual areas 28, and cuts the image in a predeterminedsize including the detected in-focus area as the thumbnail image. Morespecifically, in the case of the example shown in FIG. 3, the imagemanaging part 12 cuts the image formed of the area (for example, 4×4area) in a predetermined size including the focused object 21 as athumbnail image 29 as shown in FIG. 5.

In addition, when the in-focus value distribution above is determined,it may be considered that in the original image, there is a plurality ofin-focus areas having almost equal in-focus values dispersedly. In thiscase, for example, the image managing part 12 is allowed to perform aproper process in accordance with the descriptions of the original image20 in such a way that it cuts the image in a predetermined sizeincluding a closer in-focus area at the center of the original image asa thumbnail image among a plurality of in-focus areas having almostequal in-focus values, or cuts the image in a predetermined sizecontaining some in-focus areas as a thumbnail image among a plurality ofin-focus areas having almost equal in-focus values.

Flow Chart of the Thumbnail Image Creating Process

FIG. 6 shows a flow chart depicting the thumbnail image creating processperformed at the image managing part 12.

In FIG. 6, when a shooting function is executed by the cameraapplication 10, for example, at Step S10, and the data of the originalimage is inputted from the camera application 10 at Step S11, the imagemanaging part 12 divides the original image 20 into a plurality of equalareas 28 in accordance with the image size as the process step at StepS12, as shown in FIG. 3 described above.

Subsequently, as the process step at Step S13, the image managing part12 performs an in-focus computation for each of the areas 28 accordingto the modified Laplacian method expressed by the Equation (1), and asthe process step at Step S14, it computes the in-focus area according toa comparison process with a predetermined threshold or a detectingprocess of the maximum in-focus value, as described above.

Subsequently, as the process step at Step S15, the image managing part12 cuts the image in a predetermined size including the in-focus areadetermined in the process step at Step S14, and as the process step atStep S16, it then sets the cut image as the thumbnail image 29.

After that, the image managing part 12 stores the thumbnail image 29 inthe data storage.

Internal Configuration of the Cellular Telephone Terminal

FIG. 7 shows the schematic internal configuration of the cellulartelephone terminal according to the embodiment having the camerafunction and the thumbnail image creating function, as described above.

In FIG. 7, a data line is a transmission line which transmits variousitems of data such as sound data, e-mail data and image data.

A control line is a transmission line which transmits various items ofcontrol information, such as control data from a control part 30 whichis configured of a CPU (central processing unit).

An antenna 32 is an antenna built in the cellular telephone terminalaccording to the embodiment, which is connected to a communicationcircuit 31 for use in sending and receiving signal waves.

The communication circuit 31 converts, modulates and demodulates thefrequency of sent and received signals.

Here, the sound data of a call received at the antenna 32 and thecommunication circuit 31 is sent to a speech processing part 42, and theother received data is sent to the control part 30 and processedproperly, and then sent to each part from the control part 30 asnecessary. In addition, the received data other than the sounds of acall is the data of packet transmission. For example, the flowing datais named such as e-mail data, image data such as moving images and stillimages, music data, HTML (Hyper Text Markup Language) data, data ofprogram codes, etc.

When the antenna 32 and the communication circuit 31 supply the sounddata of a call, the speech processing part 42 decodes the sound data ofthe call, converts the decoded sound data of the call to an analog soundsignal, and sends the analog sound signal to a speaker 40 or to anearphone jack, which is not shown.

The speaker 40 is built in the cellular telephone terminal according tothe embodiment, which amplifies the supplied analog sound signal by aninternal amplifier, converts the sound signal to audible sounds, andthen outputs them outside. Thus, the speaker 40 is for outputing thesounds of the call.

A microphone 41 is built in the embodiment of the cellular telephoneterminal, which converts input sounds to an analog sound signal as wellas amplifies the analog sound signal by the internal amplifier, andsends the analog sound signal to the speech processing part 42.

When the microphone 41 supplies the sound signal of the call, the speechprocessing part 42 converts the sound signal of the call to digitalsound data and then encodes it, and sends the encoded sound data of thecall to the communication circuit 31. Thus, the antenna 32 is forsending the sound signal of the call.

A display part 33 is a display device of the cellular telephone terminalaccording to the embodiment, which has a display such as a liquidcrystal display and a display drive circuit to display images andcharacters on the screen of the display.

An operating part 34 is formed of a numeric keypad, various buttons andan operation signal generator which generates an operation signal whenthe keys and buttons are operated.

A camera part 44 corresponds to the camera function provided in thecellular telephone terminal according to the embodiment, which createsdata of a still image and a moving image obtained by converting a lightfigure to an electric signal by an imaging device, and sends the data toan image processing part 43.

Under the control of the control part 30, the image processing part 43compresses and encodes the image taken at the camera part 44, performsthe thumbnail image creating process, stores image data in the datastorage 45, and performs decompression and decoding processes for thecompressed and encoded image data read out of a data storage 45 or amemory part 35. The image after decompression and decoding and the dataof the thumbnail image are sent to the display part 33 through the dataline for display. In addition, the control part 30 may perform thethumbnail image creating process.

The memory part 35 includes a RAM (Read Only Memory) and a RAM (RandomAccess Memory). The ROM stores an OS (Operating System), a controlprogram that the control part 30 controls each part and program codesfor performing various functions, image management program codes thatthe image processing part 43 or the control part 30 performs thethumbnail image creating process, various default values, font data,dictionary data, an incoming call melody and key operating soundsaccording to the embodiment, various sound data for alarm sounds,program codes for the camera application 10 according to the embodiment,program codes for the viewer application 11, program codes forapplications to create and edit e-mail, and program codes forapplications to perform various processes for images and sounds, and inaddition to these, program codes for various applications mounted on thecellular telephone terminal, and identification information (ID) for thecellular telephone terminal, etc. The ROM includes rewritable ROM, suchas a NAND-type flash memory) or EEPROM (Electrically ErasableProgrammable Read-Only Memory), which can store e-mail data, data of atelephone book and an e-mail address book, still image data and movingimage data (video click data, etc.), key operating sounds, sound datafor alarm sounds, character data and registration data for candidatewords predictive conversion and learning data for predictive conversion,and in addition to these, various user set values, etc. The RAM storesdata at anytime as the working area when the control part 30 processesvarious data.

The data storage 45 is the storage for image data described in FIG. 1.In addition, the data storage 45 may be included in the memory part 35.

The control part 30 is formed of the CPU, which performs control overthe cellular telephone terminal, control for implementing variousfunctions according to the embodiment and various computation based onthe OS and various programs stored in the memory part 35.

In addition to this, (not shown in FIG. 7), the cellular telephoneterminal according to the embodiment of course has various functionsprovided on a classic cellular telephone terminal such as a GPS (GlobalPositioning System) function, a near wireless communication function, anoncontact communication function, a battery, etc.

CONCLUSION

As described above, according to the cellular telephone terminalaccording to the embodiment of the invention, the in-focus areaincluding the target object that a photographer intends to shoot is cutout of the original image taken by the camera function and the originalimage obtained from outside, and a thumbnail image can be created. Sincethe thumbnail image is the image that is a characteristic portion in theoriginal image, thumbnail images can be listed that are easy to see withhigh browsing and visual recognition properties even on a display devicethat has a small screen size and relatively low resolution.

In addition, the description of the embodiment is an example of theinvention. Therefore, the invention is not limited to the embodiment,which of course can be modified variously depending on the design, forexample, without deviating from the teachings of the technicalprinciples of the invention.

The thumbnail image process according to the embodiment is not limitedto the image taken by the cellular telephone terminal, which can beadapted to the image taken by other devices, and can be used for generalpurposes.

It should be understood by those skilled in the art that variousmodifications combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

1. A portable information terminal comprising: a display part which hasa screen for displaying images; an in-focus value computing part whichcomputes in-focus values from an original image that has been previouslycaptured by dividing the original image into a plurality of dividedareas and computing an in-focus value for each of the plurality ofdivided areas; an image cutting part which cuts a thumbnail image fromthe original image based on the in-focus values computed by the in-focusvalue computing part, by comparing the in-focus value for each of theplurality of divided areas with a predetermined threshold based on anin-focus value distribution, detecting an area having an in-focus valueexceeding the predetermined threshold as an in-focus area, and cuttingan image in a predetermined size including the in-focus area out of theoriginal image to set it as the thumbnail image; a data storage partwhich stores data of the thumbnail image cut at the image cutting part;and a control part which reads the data of the thumbnail image stored inthe data storage part for display on the screen.
 2. The portableinformation terminal according to claim 1, wherein the image cuttingpart detects an area having a maximum in-focus value as the in-focusarea.
 3. The portable information terminal according to claim 1, whereinthe image cutting part detects a plurality of in-focus areas in theoriginal image, and selects the in-focus area to be set as the thumbnailimage based upon criteria other than the in-focus value.
 4. The portableinformation terminal according to claim 1, wherein the image cuttingpart detects a plurality of in-focus areas in the original image, andselects the in-focus area that is closest to the center of the originalimage to be set as the thumbnail image.
 5. The portable informationterminal according to claim 1, wherein the original image is receivedfrom an outside source and stored in the data storage part prior tocomputing the in-focus value for each of the plurality of divided areas.6. The portable information terminal according to claim 1, wherein eachof the plurality of divided areas are of equal size.
 7. The portableinformation terminal according to claim 1, wherein the in-focus valuecomputing part computes the in-focus value by modified Laplaciancomputation.
 8. A computer program product for cutting thumbnail imagesfor display on a screen of a display unit, the computer program productcomprising a non-transitory computer readable medium storing programcode that is executable to provide operations comprising: computingin-focus values from an original image that has been previously capturedby dividing the original image into a plurality of divided areas andcomputing an in-focus value for each of the plurality of divided areas;cutting a thumbnail image from the original image based on the computedin-focus values, by comparing the in-focus value for each of theplurality of divided areas with a predetermined threshold based on anin-focus value distribution, detecting an area having an in-focus valueexceeding the predetermined threshold as an in-focus area, and cuttingan image in a predetermined size including the in-focus area out of theoriginal image to set it as the thumbnail image; storing data of thethumbnail image cut at the image cutting part; and reading the storeddata of the thumbnail image for display on the screen.
 9. The computerprogram product according to claim 8, wherein cutting the thumbnailimage comprises detecting an area having a maximum in-focus value as thein-focus area.
 10. The computer program product according to claim 8,wherein cutting the thumbnail image comprises detecting a plurality ofin-focus areas in the original image, and selecting the in-focus area tobe set as the thumbnail image based upon criteria other than thein-focus value.
 11. The computer program product according to claim 8,wherein cutting the thumbnail image comprises detecting a plurality ofin-focus areas in the original image, and selecting the in-focus areathat is closest to the center of the original image to be set as thethumbnail image.
 12. The computer program product according to claim 8,wherein the original image is received from an outside source and isstored in memory prior to computing the in-focus value for each of theplurality of divided areas.
 13. The computer program product accordingto claim 8, wherein each of the plurality of divided areas are of equalsize.
 14. The computer program product according to claim 8, whereincomputing the in-focus values comprises computing by modified Laplaciancomputation.
 15. A method for cutting thumbnail images for display on ascreen of a display unit, the method comprising: computing in-focusvalues from an original image that has been previously captured bydividing the original image into a plurality of divided areas andcomputing an in-focus value for each of the plurality of divided areas;cutting a thumbnail image from the original image based on the computedin-focus values, by comparing the in-focus value for each of theplurality of divided areas with a predetermined threshold based on anin-focus value distribution, detecting an area having an in-focus valueexceeding the predetermined threshold as an in-focus area, and cuttingan image in a predetermined size including the in-focus area out of theoriginal image to set it as the thumbnail image; storing data of thethumbnail image cut at the image cutting part; and reading the storeddata of the thumbnail image for display on the screen.
 16. The methodaccording to claim 15, wherein cutting the thumbnail image comprisesdetecting an area having a maximum in-focus value as the in-focus area.17. The method according to claim 15, wherein cutting the thumbnailimage comprises detecting a plurality of in-focus areas in the originalimage, and selecting the in-focus area to be set as the thumbnail imagebased upon criteria other than the in-focus value.
 18. The methodaccording to claim 15, wherein cutting the thumbnail image comprisesdetecting a plurality of in-focus areas in the original image, andselecting the in-focus area that is closest to the center of theoriginal image to be set as the thumbnail image.
 19. The methodaccording to claim 15, wherein the original image is received from anoutside source and is stored in memory prior to computing the in-focusvalue for each of the plurality of divided areas.
 20. The methodaccording to claim 15, wherein each of the plurality of divided areasare of equal size.
 21. The method according to claim 15, whereincomputing the in-focus values comprises computing by modified Laplaciancomputation.